Skip to content

feat(http): add opt-in Windows HTTP and HTTPS transport - #3994

Closed
aadi-varc wants to merge 4 commits into
boykopovar:mainfrom
aadi-varc:feat/http-winhttp-network
Closed

aadi-varc wants to merge 4 commits into
boykopovar:mainfrom
aadi-varc:feat/http-winhttp-network

Conversation

@aadi-varc

@aadi-varc aadi-varc commented Oct 11, 2026 •

Copy link
Copy Markdown

What

Blocked by upstream policy; preserved for forks. Upstream explicitly decided that a host-backed libSceHttp is not wanted on main, and closed #3844 as not planned (decision). This PR remains open pending the author’s decision to withdraw it; it is not ready for upstream acceptance under the stated policy. The complete, rebased feature remains available on aadi-varc/AnyPS5:codex/http-winhttp-fork-only.

The fork adds an experimental Windows WinHTTP transport enabled with APS5_HTTP_NETWORK=1. The default remains offline, and libSceHttp2 remains offline. This is a Windows fork feature; it does not change upstream’s offline policy.

Exports dispatch to separate offline and network implementations. The offline implementation retains current main's epoll waits, context teardown and CA-list behavior. Four commits separate handle/state management, WinHTTP transport, export dispatch, and tests/documentation.

The backend supports binary blocking requests, response headers/status/length, chunked responses, optional GZIP decoding, redirects, timeouts, concurrent independent requests and cancellation. Automatic redirects with Authorization, Proxy-Authorization or Cookie headers throw before sending. Deleting a parent invalidates and cancels descendants.

HTTPS uses Windows trust and certificate checks. Certificate-loading/CA-list APIs, epoll, enabled nonblocking operations, nonnull request-status callbacks, proxy/authentication/cookie support, explicit TLS versions and chunked uploads throw when requested. USAGE.md warns that custom-certificate initialization can stop a title before its first request. Console semantics and remaining gaps are recorded in docs/dev/debt/libSceHttp.md.

No third-party code is added. The localhost integration test generates disposable credentials with OpenSSL. Missing Python or OpenSSL reports CTest skip code 77; certificate-generation errors, timeouts and guest failures remain failures. These changes address the four implementation/reviewability points in the draft feedback.

Tested

Windows 10.0.26300 x86-64, WinLibs GCC 15.2.0 r7, Release, Python 3.12.14. Current head: 2636a48, rebased onto main 3864da595.

  • Resolved the new core/libs/CMakeLists.txt conflict by preserving upstream’s nid_shared_strings test and the HTTP offline test’s explicit environment. Documented the fork-only policy in USAGE.md.
  • Rebuilt the four HTTP/offline targets and updated dependencies. guest_offline_imports, guest_http, guest_http_response_header and guest_http_network all passed with APS5_HTTP_NETWORK=1 in the parent shell (2.67 seconds total), including real HTTP/HTTPS loopback requests. Conventions, whitespace and test-registration checks passed.
  • HTTP runtime sources remain identical to previously validated b889391, which passed the full Windows project build and five relevant tests, including patched_libraries (8.31 seconds total). The full project build was not repeated for the subsequent rebases and test-environment/documentation changes.
  • The readiness audit covered handle ownership, hierarchical deletion, concurrent cancellation, WinHTTP callback lifetime, export ABI, offline compatibility and the test harness. No blocking runtime defect was found. Missing Python/OpenSSL produce explicit CTest skips; other test failures are propagated.
  • No commercial title, PS5 hardware, Linux or macOS runtime validation. Native request-handle closure failure explicitly terminates; this exceptional OS path was not reproduced. Full upstream CI still requires a maintainer; passing checks would not override the stated policy.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in libSceHttp technical debt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source (none added)
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: none (no title bring-up validated)
  • AI-assisted: yes (Codex)

@TideOPL

TideOPL commented Oct 11, 2026

Copy link
Copy Markdown
Contributor

Some design feedback, since the draft asks for it:

  • Two bodies per export: every export now has two bodies selected by APS5_HTTP_NETWORK, and the offline branch keeps its (void)x lines after an early return in about 40 functions. A dispatch table, or moving the offline bodies into the same HttpNetwork namespace, would keep Export.cpp readable.
  • Throwing calls when enabled: with the backend on, sceHttpsLoadCert, sceHttpCreateEpoll, sceHttpSetNonblock and a non-null sceHttpSetRequestStatusCallback always throw. Many titles call sceHttpsLoadCert during init, so in practice the opt-in will abort most HTTPS titles before their first request. That follows the throw rule, but it's worth stating in USAGE.md so people know what to expect.
  • Test dependencies: guest_http_network needs Python plus an OpenSSL found next to Git for Windows. Please have it report a skip (SKIP_RETURN_CODE) rather than fail when either is missing, because CI images may have neither.
  • One large commit: about 1250 lines in one commit. Splitting it (handle and state bookkeeping, the WinHTTP transport, tests) would make it much easier to review.
  • Whether online HTTP is in scope at all is the maintainers' call.

I haven't run it yet. I'm on Windows, so I can test the WinHTTP path once it's out of draft if that helps.

AI-assisted: yes. Written with AI assistance (Claude Code); I reviewed it.

@aadi-varc
aadi-varc force-pushed the feat/http-winhttp-network branch from 9d062d1 to b889391 Compare October 12, 2026 05:55
@aadi-varc
aadi-varc marked this pull request as ready for review October 12, 2026 06:06
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 12, 2026
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 6314 docs/user/USAGE.md L84
PR 6649 docs/dev/debt/libSceHttp.md L20

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 674 docs/dev/BUILD.md
PR 1835 docs/dev/BUILD.md
PR 4891 docs/user/USAGE.md
PR 5021 core/libs/CMakeLists.txt
docs/dev/BUILD.md
docs/user/USAGE.md
PR 5390 docs/user/USAGE.md
PR 5925 docs/user/USAGE.md
PR 6044 core/libs/CMakeLists.txt
core/libs/prx/libSceHttp/CMakeLists.txt
PR 6222 core/libs/CMakeLists.txt
PR 6447 core/libs/prx/libSceHttp/Export.cpp
docs/dev/debt/libSceHttp.md
PR 6879 docs/dev/BUILD.md

This was referenced Oct 15, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

Closing: #3844 is closed as not planned, since a host-backed libSceHttp isn't wanted on main and forks can adapt the online libraries, and this PR implements that transport. A fork is the right home for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

libSceHttp: every request fails with ERROR_NETWORK; six catalog titles send HTTP requests (would a host-backed client be accepted?)

5 participants